An cloud engineer is responsible for building, designing, and maintaining cloud databases. Their role can be divided into three specialties: cloud design, development, and administration. You should have skills such as programming, data security, and storage.
Orchestrate Amazon Redshift-Based ETL workflows with AWS Step Functions and AWS Glue
July-2022 to
Description
Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. Amazon Redshift Serverless lets you access and analyze data without all of the configurations of a provisioned data warehouse.
Roles and Responsibilities
Designed and Developed ETL job on customer-reviewed data from the past 2-year data. Data is extracted, processed by the data warehouse, and transformed accurately and validated. All the process is automated with a central operating service AWS GLUE. The processed query uses various business analyses, including analyzing best-performing products, the status of delivery, etc.
A script that connects to your source data, processes it, and then writes it out to your data target. Typically, a job runs extract, transform, and load (ETL) scripts. Jobs can also run general-purpose Python scripts
Roles and Responsibilities
Create an Amazon S3 bucket to store the CSV data, and upload the CSV files to the bucket. Store the transformed Avro data in another S3 bucket, or in a data lake such as Amazon Lake Formation. Use Lambda to process and analyze the Avro data in S3, leveraging the distributed processing power of the cloud
Technologies: S3, Lambda, AWS Glue.
Industry: Information Technology
Work history
Software Developer • July-2022
Description: An AWS developer is responsible for setting up, maintaining, and evolving the cloud infrastructure of web applications. AWS developers are expected to understand the core AWS services and apply best practices regarding security and scalability.